AutoML Leaderboard
| Best model | name | model_type | metric_type | metric_value | train_time |
|---|---|---|---|---|---|
| 1_Baseline | Baseline | f1 | 0 | 1.27 | |
| 2_DecisionTree | Decision Tree | f1 | 0.877193 | 2.84 | |
| 3_Linear | Linear | f1 | 0.945455 | 3.95 | |
| the best | 4_Default_Xgboost | Xgboost | f1 | 0.973451 | 5.19 |
| 5_Default_NeuralNetwork | Neural Network | f1 | 0.948276 | 2.59 | |
| 6_Default_RandomForest | Random Forest | f1 | 0.932203 | 3.27 | |
| Ensemble | Ensemble | f1 | 0.973451 | 1.25 |
AutoML Performance
AutoML Performance Boxplot
Summary of 1_Baseline
Baseline Classifier (Baseline)
- n_jobs: -1
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
f1
Training time
0.8 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.693177 | nan |
| auc | 0.5 | nan |
| f1 | 0.67052 | 0.448688 |
| accuracy | 0.504348 | 0.448688 |
| precision | 0.504348 | 0.448688 |
| recall | 1 | 0.448688 |
| mcc | 0 | 0.448688 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.693177 | nan |
| auc | 0.5 | nan |
| f1 | 0.67052 | 0.448688 |
| accuracy | 0.504348 | 0.448688 |
| precision | 0.504348 | 0.448688 |
| recall | 1 | 0.448688 |
| mcc | 0 | 0.448688 |
Confusion matrix (at threshold=0.448688)
| Predicted as 0 | Predicted as 1 | |
|---|---|---|
| Labeled as 0 | 0 | 57 |
| Labeled as 1 | 0 | 58 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
Summary of 2_DecisionTree
Decision Tree
- n_jobs: -1
- criterion: gini
- max_depth: 3
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
f1
Training time
2.3 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.31362 | nan |
| auc | 0.924682 | nan |
| f1 | 0.877193 | 0.393939 |
| accuracy | 0.878261 | 0.393939 |
| precision | 1 | 0.85 |
| recall | 1 | 0.0782609 |
| mcc | 0.757035 | 0.393939 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.31362 | nan |
| auc | 0.924682 | nan |
| f1 | 0.877193 | 0.393939 |
| accuracy | 0.878261 | 0.393939 |
| precision | 0.892857 | 0.393939 |
| recall | 0.862069 | 0.393939 |
| mcc | 0.757035 | 0.393939 |
Confusion matrix (at threshold=0.393939)
| Predicted as 0 | Predicted as 1 | |
|---|---|---|
| Labeled as 0 | 51 | 6 |
| Labeled as 1 | 8 | 50 |
Learning curves
Permutation-based Importance
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
Summary of 3_Linear
Logistic Regression (Linear)
- n_jobs: -1
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
f1
Training time
3.3 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.163214 | nan |
| auc | 0.980944 | nan |
| f1 | 0.948276 | 0.41277 |
| accuracy | 0.947826 | 0.41277 |
| precision | 1 | 0.568079 |
| recall | 1 | 0.00812168 |
| mcc | 0.900647 | 0.568079 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.163214 | nan |
| auc | 0.980944 | nan |
| f1 | 0.948276 | 0.41277 |
| accuracy | 0.947826 | 0.41277 |
| precision | 0.948276 | 0.41277 |
| recall | 0.948276 | 0.41277 |
| mcc | 0.895644 | 0.41277 |
Confusion matrix (at threshold=0.41277)
| Predicted as 0 | Predicted as 1 | |
|---|---|---|
| Labeled as 0 | 54 | 3 |
| Labeled as 1 | 3 | 55 |
Learning curves
Coefficients
| feature | Learner_1 |
|---|---|
| PEER_PRESSURE | 1.90678 |
| YELLOW_FINGERS | 1.73776 |
| CHRONIC DISEASE | 1.69633 |
| ALLERGY | 1.68397 |
| SWALLOWING DIFFICULTY | 1.65569 |
| COUGHING | 1.60986 |
| ALCOHOL CONSUMING | 1.59961 |
| FATIGUE | 1.27991 |
| WHEEZING | 1.19506 |
| ANXIETY | 0.865616 |
| SMOKING | 0.36864 |
| CHEST PAIN | 0.244263 |
| SHORTNESS OF BREATH | 0.108942 |
| AGE | 0.10044 |
| GENDER | -0.0231852 |
| intercept | -4.98493 |
Permutation-based Importance
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
SHAP Importance
SHAP Dependence plots
Dependence (Fold 1)
SHAP Decision plots
Top-10 Worst decisions for class 0 (Fold 1)
Top-10 Best decisions for class 0 (Fold 1)
Top-10 Worst decisions for class 1 (Fold 1)
Top-10 Best decisions for class 1 (Fold 1)
Summary of 4_Default_Xgboost
Extreme Gradient Boosting (Xgboost)
- n_jobs: -1
- objective: binary:logistic
- eta: 0.075
- max_depth: 6
- min_child_weight: 1
- subsample: 1.0
- colsample_bytree: 1.0
- eval_metric: f1
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
f1
Training time
4.4 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.350546 | nan |
| auc | 0.994555 | nan |
| f1 | 0.982456 | 0.468168 |
| accuracy | 0.982609 | 0.468168 |
| precision | 1 | 0.468168 |
| recall | 1 | 0.223204 |
| mcc | 0.965809 | 0.468168 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.350546 | nan |
| auc | 0.994555 | nan |
| f1 | 0.982456 | 0.468168 |
| accuracy | 0.982609 | 0.468168 |
| precision | 1 | 0.468168 |
| recall | 0.965517 | 0.468168 |
| mcc | 0.965809 | 0.468168 |
Confusion matrix (at threshold=0.468168)
| Predicted as 0 | Predicted as 1 | |
|---|---|---|
| Labeled as 0 | 57 | 0 |
| Labeled as 1 | 2 | 56 |
Learning curves
Permutation-based Importance
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
SHAP Importance
SHAP Dependence plots
Dependence (Fold 1)
SHAP Decision plots
Top-10 Worst decisions for class 0 (Fold 1)
Top-10 Best decisions for class 0 (Fold 1)
Top-10 Worst decisions for class 1 (Fold 1)
Top-10 Best decisions for class 1 (Fold 1)
Summary of 5_Default_NeuralNetwork
Neural Network
- n_jobs: -1
- dense_1_size: 32
- dense_2_size: 16
- learning_rate: 0.05
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
f1
Training time
2.0 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.121825 | nan |
| auc | 0.990926 | nan |
| f1 | 0.973451 | 0.593571 |
| accuracy | 0.973913 | 0.593571 |
| precision | 1 | 0.593571 |
| recall | 1 | 0.000153975 |
| mcc | 0.949138 | 0.593571 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.121825 | nan |
| auc | 0.990926 | nan |
| f1 | 0.973451 | 0.593571 |
| accuracy | 0.973913 | 0.593571 |
| precision | 1 | 0.593571 |
| recall | 0.948276 | 0.593571 |
| mcc | 0.949138 | 0.593571 |
Confusion matrix (at threshold=0.593571)
| Predicted as 0 | Predicted as 1 | |
|---|---|---|
| Labeled as 0 | 57 | 0 |
| Labeled as 1 | 3 | 55 |
Learning curves
Permutation-based Importance
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
Summary of 6_Default_RandomForest
Random Forest
- n_jobs: -1
- criterion: gini
- max_features: 0.9
- min_samples_split: 30
- max_depth: 4
- eval_metric_name: f1
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
f1
Training time
2.7 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.203583 | nan |
| auc | 0.983666 | nan |
| f1 | 0.93578 | 0.58389 |
| accuracy | 0.93913 | 0.58389 |
| precision | 1 | 0.58389 |
| recall | 1 | 0.0239471 |
| mcc | 0.88495 | 0.58389 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.203583 | nan |
| auc | 0.983666 | nan |
| f1 | 0.93578 | 0.58389 |
| accuracy | 0.93913 | 0.58389 |
| precision | 1 | 0.58389 |
| recall | 0.87931 | 0.58389 |
| mcc | 0.88495 | 0.58389 |
Confusion matrix (at threshold=0.58389)
| Predicted as 0 | Predicted as 1 | |
|---|---|---|
| Labeled as 0 | 57 | 0 |
| Labeled as 1 | 7 | 51 |
Learning curves
Permutation-based Importance
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
Summary of Ensemble
Ensemble structure
| Model | Weight |
|---|---|
| 4_Default_Xgboost | 1 |
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.350546 | nan |
| auc | 0.994555 | nan |
| f1 | 0.982456 | 0.468168 |
| accuracy | 0.982609 | 0.468168 |
| precision | 1 | 0.468168 |
| recall | 1 | 0.223204 |
| mcc | 0.965809 | 0.468168 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.350546 | nan |
| auc | 0.994555 | nan |
| f1 | 0.982456 | 0.468168 |
| accuracy | 0.982609 | 0.468168 |
| precision | 1 | 0.468168 |
| recall | 0.965517 | 0.468168 |
| mcc | 0.965809 | 0.468168 |
Confusion matrix (at threshold=0.468168)
| Predicted as 0 | Predicted as 1 | |
|---|---|---|
| Labeled as 0 | 57 | 0 |
| Labeled as 1 | 2 | 56 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic